Bayesian networks with JAGS

Network structures and models

Structure

Models

\[ \text{T1D | Metabolite} \sim \text{Bern}(\text{logit}(\alpha_0 + \alpha_1*\text{Metabolite}))\\ \text{Methylation | Metabolite, T1D} \sim \text{N}(\delta+\gamma*\text{Metabolite}+\beta*\text{T1D},\sigma^2)\\ \]

Priors

\[ \alpha_0 \sim \text{N}(0,100)\\ \alpha_1 \sim \text{N}(0,100)\\ \delta \sim \text{N}(0,100)\\ \gamma \sim \text{N}(0,100)\\ \beta \sim \text{N}(0,100)\\ \sigma^2 = \frac{1}{\tau}\\ \tau \sim \text{Gamma}(0.0001,0.0001) \]

Model comparisons

DIC distribution by network structure

Candidates from cits package

Number of Pairs Where Structure Is Best
Var1 Freq
6 1
10 27
14 19
19 2
23 60
24 30 
## Warning: Removed 233 rows containing non-finite values (stat_boxplot).

Non-cits pairs

Number of Pairs Where Structure Is Best
Var1 Freq
10 1
14 60
19 2
21 3
23 3
24 31

DIC differences

name value n
Qual. Diff. Best-Second Different arc sets 2
Qual. Diff. Best-Second Different number of directed/undirected arcs 137
Qual. Diff. Second-Third Different arc sets 46
Qual. Diff. Second-Third Different number of directed/undirected arcs 93

Permutation tests

Potential problems

DIC

  • “From Bayesian perspective, DIC is not theoretically justified since it measures the fit of the model when the parameters are fixed to the posterior expectation and is not therefore an unbiased estimate of the true generalization utility.” (Piironen & Vehtari, 2017)

  • “The numerical experiments show that the over-fitting in the selection may be a potential problem and hinder the model selection considerably. This is the case especially when the dataset is small (high variance in the utility estimates) and the number of models under comparison large (large number of variables). Especially vulnerable methods for this type of overfitting are CV, WAIC, DIC and other methods that rely on data reuse and have therefore relatively high variance in the utility estimates.” (Piironen & Vehtari, 2017)

Priors

Tim was wrong about gamma priors causing problems. This issue is specific to logistic models with random effects, where the prior is \(b_i \sim N(0,\frac{1}{\lambda})\). The Piironen paper can be a little unclear, so maybe we need a better resource.

References